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ABSTRACT 

The summer school sessions that colleges offer their undergraduates are sometimes 
considered supplementary activities and are rarely perceived as central to a college’s mission or 
effectiveness. However, analyses of college transcript data that tracked a nationally- 
representative sample of undergraduates for several years and through multiple colleges show 
that those undergraduates who attend summer school at the end of their first year of college have 
better retention rates thereafter and are significantly more likely to complete a degree. This 
relationship remains statistically significant and of substantial size after controlling for student 
socio-demographic characteristics and for their academic performance prior to taking summer 
school, using propensity score matching methods. Moreover, students with lower academic 
performance and others at higher risk of dropping out who attend summer school also have 
higher graduation rates, suggesting that administrators should conceive of summer school 
enrollment as an important tool for improving undergraduate retention and degree completion. 
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INTRODUCTION 

Rates of college-going are high in the US: nationwide about 62% of Americans aged 
between 25 and 29 have some post-secondary education (Aud et al., 2012, p. 286). 

Unfortunately, high rates of access do not translate into high rates of graduation: only 26% of 
students entering community college degree programs and 63% of undergraduates entering 
public four-year colleges complete a degree within six years (Becker et al., 2007). This has led to 
calls from government officials and philanthropies for improvements in degree completion rates 
(Kanter et al. 2011; Lumina Foundation, 2013). 

Researchers have identified multiple factors associated with failure to complete a college 
degree, including poor high school academic preparation, inadequate financial aid, and the 
burden of family or work responsibilities (Bozick, 2007; Braxton, 2000; Kuh et al., 2010; Pema, 
2010; Seidman, 2005; John et al., 2010; Tinto, 1993). Many of these factors are relatively 
intractable: undergraduates who work to pay their bills often cannot afford to reduce their work 
hours; students with parenting obligations are unlikely to escape those responsibilities; and large 
increases in the level of federal financial aid for undergraduates are unlikely in an era of 
economic cuts. Policy-makers therefore look for interventions beyond these factors that might 
have a substantial impact on degree completion. 

Educators have undertaken many college-based interventions and reforms aimed at 
improving graduation rates including changes in curricula, scheduling, support services for 
students, and mode of delivering courses (Rice &Taylor, 2003; McCormick et al., 2011). Some 
college-based interventions have shown large treatment effects: for example, the City University 
of New York’s ASAP program increased associate degree graduation rates by 28 percentage 
points through a mix of block scheduling, smaller classes, intensive counseling and tutoring, and 
modest financial incentives for students (Linderman & Kolenovic, 2012; Scrivener et al., 2012). 
However, multifaceted college interventions like ASAP tend to be expensive. ASAP costs an 
additional $6,612 per FTE student per year (Levin & Garcia, 2012, p.15) compared to a regular 
annual expenditure per FTE community college student of $7,650 nationwide (College Board, 
2012). Consequently, the search continues for less expensive interventions with high impact on 
college graduation rates, so-called “low hanging fruit.” 

This paper examines one long-standing feature of college life - summer school - that is 
not usually viewed by policy makers and education administrators in terms of retention and 
degree completion. Many colleges offer an optional summer program for undergraduates. 
Summer courses usually meet for fewer weeks than quarters or semesters in the rest of the year. 
To balance the fewer weeks, summer courses often meet more frequently or for longer hours, 
providing a more intensive learning experience compared to regular term time. 

Using national data and employing an analytical technique that minimizes selection 
effects and controls for the influence of prior academic performance and other student variables, 
the analyses reported below find 7 and 11 percentage point advantages in graduation associated 
with attending summer school after the freshman year of college, at four-year and two-year 
colleges respectively. Moreover, summer school attendance is associated with higher graduation 
rates among several types of students for whom college completion is most problematic. This 
implies that summer school should not be perceived as a supplementary or peripheral activity, 
but instead be recognized as a potentially important and relatively inexpensive tool in colleges’ 
efforts to raise graduation rates for undergraduates at risk of failure. 
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PRIOR RESEARCH AND THEORETICAL FRAMEWORK 

There has been very little research about summer enrollment during college, and those 
few studies were based on a single dataset. Using student transcript data from the NELS88/2000 
survey, Adelman (2006) found that taking courses during the summer was associated with higher 
odds of graduation. He concluded that “Summer-term enrollment works for just about 
everybody” and that “[For African-American students] earning more than four credits in summer 
terms offers a stunning boost, narrowing the completion gap vis a vis white students from 15.5 
percent to 6 percent” (Adelman, 2006, p.90 & p. 93). 

Adelman (2006) employed logistic regression models that did not address selection. 
However, as documented below, students who take summer coursework differ on background 
characteristics compared to counterparts who do not enroll in the summer. In a reanalysis of the 
same NELS transcript data, Author (2011) used propensity score matching methods to reduce 
selection bias and examined the links between attending summer school immediately after the 
first year of college and subsequent graduation rates, finding a statistically significant 
relationship between summer attendance and higher graduation rates, though of smaller 
magnitude than that reported by Adelman. 

The theoretical framework that motivated those two studies involved the concept of 
academic momentum. That concept suggests that, even after controlling for students’ high school 
preparation and demographic characteristics, undergraduates’ experiences in college during the 
first year set a trajectory for future success. It emphasizes that the manner in which a student 
progresses during that initial period - the numbers of credits attempted and completed, and the 
trajectory of grades earned - is important for retention and degree completion. In essence, 
students who are able to accumulate a full load of credits each semester and attain good grades in 
their initial semesters, or whose initial grades trend upward, are likely to persist and complete 
their course of study (Adelman, 2005; Adelman, 2006). Those who do not are less likely to 
complete a degree. Within a momentum framework, summer school is important because it 
allows students to gain additional credits towards the required 60 or 120 credits for an AA or BA 
degree, respectively; summer attendance moves a student closer to the finishing line. 

College can also be conceptualized as a set of organizational hurdles. Certain courses 
cannot be taken unless a student has already passed other prerequisite courses. In addition, 
students may be unable to take certain courses during the regular school year because of 
scheduling issues or because they are oversubscribed, and may take that course in the summer 
instead, in order to ‘catch up’ (Dainow, 2001; Taylor, 2003; Kretovics et al. 2005) Many 
colleges require entering undergraduates to take skill or placement tests and direct students with 
low scores into sequences of remedial or developmental courses (Jaggars & Hodara, 2011). 
Students may be advised not to proceed into non-remedial courses until they have passed 
remedial requirements. For some students in this situation, summer school enables them to take 
courses they need to “get back on track.” 

Another emphasis, one compatible with the momentum framework, is advanced in the 
report titled “Time is the Enemy” by the organization ‘Complete College America’ (2011). It 
notes that four of every ten undergraduates attend part-time, and that three-quarters of 
undergraduates are “juggling some combination of families, jobs and school while commuting to 
class.” The result is that “Time is the Enemy of Completion” and “the longer [college] takes, the 
more life gets in the way of success.” Their prescription is to speed up the progress of non- 
traditional students. 
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Attending summer school fulfills this agenda in two ways. It allows students to 
accumulate more credits, but it also avoids a situation where students put aside studying for the 
summer months and shift their efforts towards other competing obligations: earning money, 
spending time with family, and so on. If, as the report avers, there is a continual competition for 
students’ attention between education and other life activities, then attending summer school acts 
to sustain an academic focus and to hold off those other competing demands. 

Another possibility is that the intensive class scheduling of summer session courses may 
be pedagogically more effective for some students. Attending more class sessions per week may 
help in retaining ideas and building competence. Some students may find it easier to absorb 
material while taking fewer courses (during the summer) than the multi-class course load in a 
regular school year semester. Thus summer attendance may improve their academic performance 
or morale. 

Finally, there are certain conceptual parallels between undergraduates attending summer 
school and previous research on summer learning in K-12 education. Numerous studies have 
noted large gaps in educational progress between social classes that widen during the summer 
breaks during K-12 schooling (Mumane, 1974; Heynes, 1978; Entwhistle &Alexander, 1992; 
Entwhistle &Alexander, 1994; Cooper et al., 1996; O’Brien, 1998; Downey et al., 2004). This 
well-documented phenomenon known as “summer fallback” refers to a pattern whereby low- 
SES students fall further behind their more privileged peers over time, not because of differences 
in the rate of learning during the school year, but largely because of a large gap in the amount of 
learning taking place over the summer. If similar dynamics were to occur for college-age 
populations as well as for high school students, one would expect to find that academic skills 
would atrophy over the summer months among lower-SES undergraduates, while such skills 
would be preserved over the summer among higher-SES students. 

How might enrolling in summer school at college relate to the fallback phenomenon? 
Comparing students who attend summer school to otherwise similar students do not, one 
hypothesizes a benefit gained by attending summer school which should be largest for 
disadvantaged students, because relatively-privileged students accrue or maintain knowledge and 
skills even when they do not attend college, while relatively disadvantaged students may suffer 
skill setbacks in summer unless they remain in school. Furthermore one would expect that 
summer school would have a larger effect on students in institutions where lower SES students 
predominate, compared to colleges serving more affluent or higher SES students. 

Prior studies and this theoretical framing lead to the following research questions: 

1. What kinds of students currently opt to take summer coursework? Is there evidence of 
selection? 

2. After correcting for student background characteristics and selection, is summer attendance 
after the first year of college associated with better retention and graduation prospects? 

3. Do differences in outcomes associated with summer attendance vary across institutional type, 
comparing community college students with four-year college students? 

4. Within each type of college, is there heterogeneity of effects: what kinds of students evidence 
larger differences in educational outcomes than others, related to summer attendance? 

These questions are answered using a different source of data than that used in prior studies. 
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DATA & METHODS 

The National Center for Education Statistics, a division of the US Department of 
Education, funds a longitudinal survey that follows a nationally-representative sample of college 
freshmen for six years after they first enter college. The latest complete cohort of this survey, 
known as the Beginning Post-Secondary Survey or BPS, followed a sample from 2003 until 
2009. Methodological details are available on-line at http://nces.ed.gov/survevs/bps/about.asp . 

As a supplementary project, the BPS staff requested transcripts from all the colleges and 
universities that each student reported they had attended, and coded courses taken, grades 
received, and exact dates were spent in college. 

The following analyses of the BPS transcripts are limited to students who entered into a 
degree program (AA or BA) in academic year 2003-2004. Only those students attending either a 
public institution or a private non-profit college are included. Consequently, students at 
proprietary or for-profit colleges are not present in these analyses. The BPS provides panel 
weights in order to account for attrition and non-response and replicate weights to correct 
standard errors for the survey’s multi-stage sampling design. Those weights were used in the 
models reported below. The BPS also replaces missing data through a ‘hot-deck imputation’ 
procedure. 

The main independent or ‘treatment’ variable is dichotomous, and indicates whether or 
not a student enrolled in classes during the summer immediately after his or her first year of 
college. Summer school attendance was identified from the transcript start-and end-dates of 
courses. Any term that started in May or later and ended by September was treated as a summer 
school session. The focus is only on summer school taken immediately after the first year of 
college. Some students also take summer courses in later years in college, but the analyses in this 
paper do not address that issue. 

Two ki nds of statistical models were estimated in order to understand the association 
between summer school attendance and educational outcomes. The first is multivariate logistic 
regression. Although the logistic models include many control variables representing potential 
confounds, their estimates of the coefficient for summer school attendance will be biased to the 
extent that students who attend summer school differ from students who do not attend summer, 
in terms of academic, demographic, or other characteristics. (This is termed selection bias.) In 
order to minimize selection bias, a set of propensity score matched models was also estimated. 
Propensity-score adjustments can reduce the bias of estimates due to observables and provide a 
more accurate inference about any ‘treatment effect’ that may exist (Morgan & Harding, 2006; 
Guo & Fraser, 2010; Morgan & Winship, 2007; Reynolds & DesJardins, 2009; Shadish et al., 
2002 ). 

Propensity-score matching proceeds in four stages. First a logistic or probit model is run 
to predict who undertakes the ‘treatment’ - in our case who attends summer school after their 
first year in college. This ‘treatment model’ or ‘propensity score model’ contains all available 
variables including students’ demographic background, their high school academic 
characteristics, their college’s characteristics, and the student’s academic performance 
throughout their first year of college. The dependent variable is a dummy variable, with a value 1 
for attended summer session, and zero otherwise. Interaction terms are included in the treatment 
or propensity model. Including predictors that are multicollinear and predictors that are not 
statistically significant are not problematic for this stage of analysis, since the goal is to 
determine the predicted probability of treatment, rather than the coefficients of individual 
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predictors (Shadish et al., 2002, p.162). A complete list of predictors used in the propensity 
score model is provided in the Appendix. 

From the logistic model one calculates for each student the probability of taking summer 
school, given their values on all covariates in the model, a quantity known as the propensity 
score. The second step involves matching persons who did attend summer school with persons 
with almost identical propensity scores but who did not attend summer school. This was 
accomplished through a STATA program called ‘psmatch2’ that undertakes a form of matching 
known as nearest neighbor matching with a caliper (Leuven & Sianesi, 2012). In this instance 
each treated case is matched with three untreated cases whose propensity scores are numerically 
very close to the treated case’s score; the conventional distance or ‘caliper’ we used is within one 
quarter of a standard deviation of the treated case’s propensity score (Guo & Fraser, 2010, 
p.147.) The end result is a treatment group and a control group, containing persons matched on 
the propensity score for treatment. 

In propensity score matching, there are typically some extreme cases that cannot be 
matched. Some individuals have attributes that make it extremely likely that they will attend 
summer school; they have a very high propensity score as a result. In principle, there might be no 
individuals with very high propensity scores who did not attend summer school, so there are no 
untreated cases at this end of the scale to match to the treated cases. Conversely, some 
individuals may have a very low propensity to take summer school. At this extreme there may be 
many cases who did not attend summer session, but perhaps no equally-low scorers who did 
attend summer session. Hence some cases at this extreme cannot be matched. 

Propensity score matching and the logic of the Counterfactual model apply to an area of 
overlap where matching is possible; the term used for this is ‘common support.’ If large 
proportions of cases could not be matched, that would raise the issue of whether the matched 
sample was representative of the larger population of undergraduates (Shadish et al., 2002, 
p.164). However, in our analyses, as reported in the Appendix, matching proved possible for 
well over 99% of cases, so this is not a problem. A report of the percent matched is provided 
there for each covariate. 

In experimental studies, random assignment of cases into treatment and control groups 
ensures that the two groups are balanced in terms of observed and unobserved background 
characteristics. In a parallel fashion, successful matching on propensity for treatment in a non- 
experimental study should result in two groups that are closely balanced on all observed 
characteristics (though not necessarily on unobserved attributes). Consequently it is important to 
ascertain whether matching did in fact result in balance on covariates (Morgan & Harding, 

2006). The Appendix to this paper provides several statistics assessing the degree of balance. 

The term ‘mean bias’ refers to the average of the differences between treatment and control 
group on all covariates, each measured in decimal fractions of a standard deviation, after 
matching. In our analyses biases were very small, never exceeding 0.05 s.d., indicating a good 
match. The column titled ‘% reduction in mean bias’ indicates the percentage reduction in that 
bias, comparing before and after matching. As the Appendix reports, matching reduced bias by 
99% or more; in other words, the treatment and control groups became much more balanced than 
the unmatched sample. Yet another indicator of the quality of the balance is the p value of a t-test 
comparing the mean value of the treatment group and the control group on each covariate, 
following matching. If the matching procedure is successful there should be no statistically 
significant differences on any co variates between treated and untreated groups. (So p values 
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close to 1.0 are desirable.) On these criteria, the matching in our analysis was effective in 
reducing selection biases. 

The third step of a propensity-score matched analysis determines the magnitude of 
treatment effects and their statistical significance. This is accomplished in the psmatch2 program 
by calculating a t-test between the treated and untreated groups and reporting estimates for the 
“average effect of treatment on the treated” or ATT. 

A fourth step involves a sensitivity analysis of the ATT results. Propensity score 
matching cannot rule out the possibility that there might still be unmeasured or unobserved 
variables which are associated with attending summer school. A sensitivity analysis answers the 
question: how large would an effect from an unmeasured hypothetical variable need to be to 
render the results just obtained from a propensity analysis statistically non-significant? The 
STATA program “mhbounds” was used to calculate this quantity and report and interpret 
sensitivity values in the findings section below (Becker & Caliendo, 2007). 

Causation and the Counterfactual Model of Causal Inference 

The use of terms such as ‘treatment effect’ and ‘benefit from attending summer school’ 
may raise concerns that these terms imply causation and violate the credo that ‘association is not 
causation.’ In recent years, however, statisticians have argued that under certain carefully 
specified conditions, observational data can be used to test causal claims and not just association 
(Guo & Fraser, 2010; Pearl 2000; Shadish et al., 2002). When certain conditions are met, 
including achieving statistical balance on a range of substantively important covariates, scholars 
have demonstrated that the Counterfactual Model of Causal Inference, of which propensity score 
matched models are one application, does permit researchers to make causal inferences, so long 
as limitations are acknowledged (e.g., Shadish et al., 2002 p.161-163; Morgan & Harding, 2006; 
Morgan & Winship, 2007; Morgan & Todd 2008; Reynolds & DesJardins, 2009). 

FINDINGS 

The findings for degree-seeking students at community colleges or two-year institutions 
contrast in certain respects with those for entrants to four-year colleges, so the following 
presentation of results is divided by the type of college a student first entered. 

1. What kinds of community college students attend summer school? 

Table 1, which reports descriptive statistics for the sample, indicates that about 30% of 
community college students seeking degrees enroll in summer courses at the end of their first 
year of college. Table 2 provides a multivariate analysis of attendance in summer school for 
these two-year college undergraduates. Model 1 in Table 2 reveals that female students are more 
likely to take summer courses than males, and that Asian and Black students have a higher 
likelihood of taking summer school than Whites. Models 2 and 3 of Table 2, suggest that these 
demographic differences are not reflections of differences in academic preparation in high school 
or even academic performance during the first year of college; their coefficients change little 
after controlling for those factors. 
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However, prior academic performance in high school does affect the odds of attending 
college during the summer (Table 2, Model 2). Ceteris paribus, students who had lower GPAs in 
high school are less likely to take summer school. Similarly, students who performed well 
academically during their first year at community college were more likely to enroll in the 
summer (Table 2, Model 3). Two performance measures were included: cumulative college GPA 
prior to the summer, and cumulative college credits earned, prior to the summer. Both had 
statistically significant associations with summer school attendance. 

This pattern is consistent with Adelman’s notion of academic momentum: those students 
who initially thrive in college, gaining good grades and credits, are drawn into greater levels of 
involvement, while students who struggle in terms of accruing credits and good grades are less 
likely to increase their level of involvement and may decrease their academic momentum. 

Students who had enrolled in a bridge course between high school and starting college 
had nearly double the odds of enrolling in summer classes after the end of their first year of 
college. This is noteworthy because summer bridge programs at community colleges often enroll 
academically-challenged students, and are often focused on improving basic academic skills; yet 
taking a bridge program before formally starting college seems to lead such students to enroll 
once again in summer courses one year later. 

2. What outcomes are associated with attending summer school for students at two-year 
colleges? 

Table 3 summarizes a series of conventional regression models and propensity-score 
matched models predicting important milestones in the academic careers of two-year college 
entrants. They indicate whether attending summer school at the end of one’s first year in college 
is associated with better outcomes on these milestones. The conventional regression analyses on 
the left of the table do not correct for selection bias - the fact that students who attend in the 
summer are systematically different from those who do not attend. The right hand of the column 
presents propensity matched models that do correct for selection bias. 

All of the effects reported in Table 3 came from regression models controlling for the 
many covariates or potential confounds including: race/ethnicity, gender, age, single parents, 
US-born, citizenship, student’s marital status, having a dependent, whether the student’s native 
language is English, household size, mother and father’s educational level, income, home 
ownership and investments. There were also measures of students’ high school academic 
preparation: student’s high school GPA and numbers of credits earned, their SAT quartile and 
highest level of mathematics taken in high school, and whether they earned any college credits 
while in high school; whether they lacked a regular high school diploma. The models in Table 3 
also controlled for the following college characteristics: total enrollment, percent of students who 
received federal grant aid, and loan aid, tuition quartile; percent of Black and Latino students at 
the institution. Finally, several control variables reflected student academic progress and 
engagement during the freshman year of college, prior to tasking summer school: whether the 
student viewed themselves as primarily a student working to pay expenses, or primarily a worker 
who was taking courses in college; whether the student had paid work during the summer; 
whether the student registered full-time or part-time; the student’s cumulative credits earned and 
cumulative GPA during the first year prior to summer school; the ratio of credits completed to 
credits attempted in the first year of college; and whether the student took a summer bridge 
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course prior to the Fall semester of their first year. These same variables were used to predict 
‘treatment’ in the propensity matched models in Table 3. 

Before detailing the findings in Table 3, however, a potential logical pitfall should be 
discussed. In principle, summer attendance could be a symptom rather than a cause of retention. 
Imagine that a student was about to drop out at the end of the first year. That student might be 
less willing to enroll in summer school at the end of the first year. Hypothetically, therefore, the 
apparent association of summer attendance with later academic outcomes such as graduation 
might be spurious, and really reflect the fact that students who are about to drop out of school are 
much less likely to take summer school at the end of the first year. 

To avoid this problem, all the results analyzed in Table 3 are conditioned on a student 
returning to college for the Fall term of their second year: the analyses only include students who 
did return after the summer and register for the Fall term of the sophomore year. This 
conditioning makes a substantial difference to the findings. The apparent effects of summer 
enrollment were much larger in models that were not conditioned on Fall 2 nd year enrollment 
(models not shown). However, the conditional effects reported in Table 3 remove this potential 
source of spuriousness, and still indicate that there are summer school effects. This is the most 
conservative approach. 

Table 3 summarizes six different analyses and focuses on the effect sizes for three 
different outcomes, each of which can be viewed as a milestone in a student’s career. The first 
such milestone, in the first row of table 3, reports whether a student remained enrolled in the 
Spring of their second year. The conventional regression model reports that 86% of those who 
took summer school came back for Spring of their second year, compared to 79% of students 
who did not take summer school, a highly significant difference. Continuing across this first row 
of Table 3, the propensity model finds that 88% of summer attendees returned to college for the 
Spring of their second year, compared to 85% of those who did not attend in the summer. The 
propensity score model did not show a statistically significant effect, implying that the observed 
difference in reenrolling is partly attributable to background differences between treated and 
untreated students. A second row in Table 3 shows a similar pattern in estimating the effect of 
summer school on stopping out of college: conventional regression indicates that summer school 
reduces stopping out, but propensity analysis suggests this is due to selection. 

The most important milestone is whether a student graduated with a degree (AA or BA) 
within six years of entering a two-year college. The third row of Table 3 shows that students who 
attended summer school had a 9.79 percentage point higher graduation rate in the conventional 
regression model. They also had a 10.98 percentage point higher graduation rate in a propensity 
matched model. Both effects were statistically significant at the 0.001 level. This substantial 
effect of summer school on degree completion is estimated after controlling or adjusting for high 
school academic background, and performance in the first year of college, and also for students’ 
demographic characteristics. 

The next question to be considered is whether these benefits occur equally for all kinds of 
students in two-year colleges, or whether summer school attendance has a larger effect size for 
certain types of students than for others. 

3. Heterogeneity in summer effect sizes among two-year-college students. 

Heterogeneity in the effects of taking summer school was examined for several 
subgroups of two-year college students, dividing on age; race; gender; family background; high 
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versus low college GPA; whether or not the student had taken a remedial class; and for high 
propensity versus low propensity to take summer courses. In each case separate propensity-score 
matched models for each subgroup determine (a) whether there is a statistically-significant effect 
of summer for one or both groups and (b) to compare the result- or effect-sizes across the groups. 
The outcome variable used was graduated with any degree (AA or BA) within six years. 

The results of these analyses are reported in Table 4. Comparing students younger than 
21 with those 21 or older, attending summer school at the end of one’s first year in college was 
significantly associated with higher rates of graduation for both age groups. The size of the effect 
was roughly comparable in both age groups. 

To examine racial and ethnic subgroups, limited sample sizes constrained the researchers 
to compare Black and Latino students on the one hand with White, Asian and ‘Other Race’ 
students on the other. Statistically significant positive effects of summer school were found for 
both race/ethnic groups. The size of the summer effect was not significantly different across 
those racial groups. 

Summer school had a positive association with graduation for both men and for women 
students, of roughly the same magnitude. A similar pattern where both groups benefited about 
equally was found when comparing students who were 1 st generation college goers with students 
whose parents had attended college. 

Two dimensions related to prior academic performance are examined in Table 4. The first 
contrasted cumulative GPA in college during the first year (prior to summer school). Summer 
school attendance had a statistically significant positive association with graduation for both 
higher and lower GPA students. Students were also contrasted according to whether or not they 
had taken any remedial courses during their first year. Again, both remedial and non-remedial 
students showed a large positive statistically-significant association between summer school and 
graduation. 

Finally, analyses examined whether students with a high probability to enroll in summer 
school (based on a multivariate model that contained academic, demographic, and other 
predictors) exhibited the same graduation boost as students whose profile suggested they were 
unlikely to take summer school. For both high propensity and low propensity score students, 
attending summer school was significantly associated with higher chances of graduation. 

In sum, heterogeneity was considered on several dimensions: race, sex, age, family 
background, and academic performance. For all subgroups of two-year college students there 
was a statistically significant higher graduation rate associated with attending summer school. 
The effect sizes were not statistically different across subgroups. 

Turning to analyses of summer school for students entering four-year colleges, results 
contrast in important respects with the findings for two-year-college entrants. 

4. Which four-year college students attend summer school? 

Nationwide, about 21% of four-year college undergraduates enrolled in the summer after 
their first year, a smaller incidence than among community college students (Table 1). Table 5 
indicates that Asian students at four-year colleges have nearly double the odds of enrolling in 
summer school than those of whites, even after holding numerous other variables constant. Black 
students are also more likely to enroll in summer school compared to whites. Married students 
also have statistically significantly higher odds of summer enrollment. Four-year college students 
whose native language is English are considerably less likely to take summer school than non- 
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native English speakers, and students with more need-based aid are less likely to enroll in 
summer school. 

Among four-year college students, a stronger high-school academic performance as 
indicated by a higher GPA is associated with a lower probability of enrolling in summer school. 
However, these effects cease to be significant after controlling for students’ academic 
performance during their first year of college. The higher ones first year college GPA, the more 
likely one is to enroll in summer school. 

As was also the case for community colleges, those students who attended a bridge 
program in the summer prior to enrolling in college were much more likely to enroll in summer 
courses at the end of their first year of college: their odds are 2.7 times as high as non-bridge 
students after controlling for other covariates. 

5. What outcomes are associated with attending summer school for four-year college 
students? 

Table 6 reports the correlates of summer school for four-year college students, looking at 
the same milestones or outcomes as before, and with similar controls. Several outcomes were 
not statistically significant. However, in both the conventional regression and in the propensity 
model, there was a statistically significant effect of summer school upon six-year graduation. 

The magnitude was 3.74 percentage points in the regression model and 6.93 percentage points in 
the propensity model. 

6. Heterogeneity in summer effect sizes among four-year undergraduates. 

Table 7 reports on heterogeneity in the relationship between summer school and 
graduation within six years. Statistically significant positive relationships between summer 
attendance and the outcomes were found for most subgroups. One exception was that summer 
school attendance was not associated with better graduation chances for students who had a GPA 
above the median in their first year of college. It was only positively associated with students 
who had GPAs below the median in the first year. Taken at face value this suggests that 
academically high performing students in four-year schools do not benefit from taking summer 
school, but that summer school is associated with significantly better graduation prospects 
among academically lower-performing four-year college students. 

Older students in four year colleges did not seem to have a significant summer school 
effect and nor did minority students. Some caution is warranted in interpreting these latter 
effects, because in both cases, the numbers of cases for the subgroup were smaller, raising issues 
of statistical power, and because the effect sizes for minorities (for which p=0.089) were not 
statistically different than those for non-minority students, similarly for older versus younger 
students. 

7. Sensitivity Analyses. 

In Table 8, for community college students, a non-parametric Mantel and Haenszel test 
indicates that an unobserved variable would have to change the odds of treatment by a factor 1.5 
for the observed effect of summer attendance to be rendered statistically non-significant. Since 
that would be a quite strong unobserved bias, it seems unlikely that the treatment effect of 
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summer for community college students that we observed is caused by an unmeasured 
confounder. The same test indicates that for four year college students, a hypothetical 
unobserved bias that changed the odds of treatment by a factor of 1.15 would negate the 
observed significant result. An effect of that magnitude suggests that some degree of caution is 
merited in interpreting the observed impact of summer school among four-year college students, 
although one should note that a sensitivity analysis does not imply that such a bias actually 
exists; it simply tests how large a hypothetical bias would need to be before observed results 
were rendered not significant. 

DISCUSSION AND CONCLUSION 

Based on prior research and insights from studies of summer fallback, the researchers 
anticipated that undergraduates who attend summer school after the end of their first year in 
college might have higher graduation rates than their academically and socially similar 
counterparts who do not attend summer school. This study supported prior claims made on the 
basis of different longitudinal data: summer school attendance after the freshman year of college 
was associated with 7 and 11 percentage point advantages in graduation, at four-year and two- 
year colleges respectively. Among the many factors controlled for in these analyses was 
academic performance during the first year in college, as well as various aspects of family 
background. So these effects of summer enrollment were not simply consequences of students' 
stronger academic backgrounds or prior performance. 

Scholars argue that findings from properly-balanced propensity-score models may be 
read as evidence for a causal relationship, and not just evidence of statistical association (Guo & 
Fraser, 2010; Morgan & Winship, 2007). The effects reported here are statistically significant 
and of considerable magnitude, suggesting that summer school may serve as one way to enhance 
graduation rates. Colleges might consider ways of encouraging more of their students to enroll in 
summer school. Most especially, community colleges would show the largest gains in graduation 
if the results reported here are generalizable to interventions designed to pull more students into 
summer school. One such intervention - a Randomized Control Trial - is currently underway at 
the City University of New York, but it will be some years before outcomes such as graduation 
can be assessed. 

Finally, one should note that federal policy regarding the provision of summer school has 
been fluctuating and has recently backed away from supporting summer school attendance. The 
Higher Education Opportunity Act of 2008 provided, for the first time, a Year-Round Pell Grant 
that became effective in academic year 2009-2010. Nationwide, about 800,000 undergraduates 
took advantage of that provision to pay for summer school in that year. However, this year-round 
or summer Pell grant was subsequently eliminated by a federal statute, the Department of 
Defense and Full-year Appropriation Act of 2011, following testimony by the Secretary of the 
US Department of Education that the extra grants "cost 10 times more than anticipated and failed 
to demonstrate a meaningful impact on students' academic progress" (Kantrowitz, 2011). It is 
unclear what evidence underlay that statement about lack of academic progress. Several studies 
cited above, along with the research reported here, suggest that summer school may indeed have 
‘a meaningful impact’ on undergraduates’ progress, and one hopes that educational policy¬ 
makers will become aware of that evidence. 
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APPENDIX 

Table 1. Descriptive Statistics of BPS sample (weighted) 


2 year college 4 year college 


students in degree students in degree 

programs _ programs _ 

Mean Std. Err. Mean Std. Err. 


Treatment 

% attending summer school after one year 
Covariates 

Demographic Variables 
Female 
Age 
White 
Black 
Latino 
Asian 
Other 

Student is Married 
Have Any Dependent 
English as Primary Language 
Parental Education Variables 
Parents' Education is Unknown /Less than High 
School 

Parents graduated High School 
Parents' Education has less than BA 
Parents have BA or higher 
SES Variables 

Parents/Student Owns a home 
HH Income (log) 

Total amount of need-based grant aid 2003-2004 
Academic Variables 
High School GPA 
No GPA info 
.5-1.9 (low GPA) 

2.0-2.9(mid-low GPA) 

3.0-3.4 (mid-high GPA) 

3.5-4.0(high GPA) 

Remediation 

Didn’t take any remediation 
Took one or more remediation 
Cumulative GPA up to First Summer 
Cumulative Credit Earned up to First Summer 
Taking Summer Bridge Program 
Full-time status in Fall 2003 
Full-time status in Spring 2003 
N 


.301 

.013 

.210 

.007 

.575 

.015 

.556 

.009 

22.888 

8.426 

19.200 

4.084 

.615 

.022 

.699 

.013 

.139 

.012 

.097 

.012 

.152 

.013 

.098 

.006 

.041 

.005 

.057 

.003 

.050 

.004 

.047 

.004 

.137 

.0 

08 

.025 

.003 

.217 

.010 

.037 

.004 

.883 

.008 

.899 

.006 

.105 

.008 

.040 

.003 

.304 

.011 

.174 

.006 

.297 

.013 

.218 

.007 

.292 

.010 

.566 

.008 

.696 

.016 

.839 

.006 

10.164 

.1.999 

10.749 

1.551 

920.849 

31.274 

2591.405 

81.716 


.317 

.011 

.082 

.006 

.041 

.006 

.010 

.001 

.271 

.011 

.138 

.007 

.245 

.010 

.316 

.008 

.124 

.009 

.451 

.009 

.685 

.011 

.825 

.006 

.314 

.011 

.174 

.006 

2.521 

1.179 

2.816 

.869 

14.337 

10.316 

25.077 

9.033 

.092 

.008 

.060 

.006 

.356 

.012 

.796 

.011 

.490 

.017 

.863 

.008 

4,280 


7,390 



Note: For continuous variables (age, income, cumulative GPA up to first summer, cumulative credit earned up to first summer, 
and ratio of credit eamed/attempted up to first summer), a standard deviation is reported instead of a standard error. 

Data Source: NCES Beginning Postsecondary Study 2003/2009 cohort, PETS transcript data. 
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Table 2. Attendance in Summer Courses after the first year of college: Degree-program entrants 
to Two-year Colleges (weighted) 


Odds ratios from logistic regressions predicting First Summer Attendance. 


Variables 


( 1 ) ( 2 ) 
Demographic Academic 
and SES Background 

in High 
School 

(3) 

Academic 
Background 
in College 
up to First 
Summer 

Black {Ref: White) 


1.290* 

1.322* 

1 773 *** 

Latino 


1.029 

1.013 

1.194 

Asian 


2.811*** 

2.649*** 

2.499*** 

Other race 


0.548** 

0.537** 

0.620* 

Female 


1.456*** 

1.451*** 

1.389*** 

Age (centered) 


1.042** 

1.022 

1.011 

Age A 2 (centered) 



0.999* 

0.999 

0.999 

HH Income (log) 



0.969 

0.973 

0.955 

Parents' Education is Unknown 

{Ref: Parent HS) 


0.914 

0.909 

0.921 

Parents' Education is less than BA 


1.075 

1.057 

1.013 

Parents have BA or higher 



1.135 

1.112 

1.005 

Home Ownership 



1.108 

1.143 

1.092 

Total amount of need-based grant (in thousand$) 


1.046* 

1.047* 

.997 

Married 



1.185 

1.179 

1.033 

Have Any Dependent 



1.076 

1.052 

1.242 

English as Primary Language 



0.879 

0.889 

0.883 

No High School GPA Info ( Ref:gpa3.0-3.4) 


7 

1.209 

1.375 

Low HS GPA (0.5-1.9) 



A / 

0.468** 

0.550* 

Medium-low HS GPA (2.0-2.9) 


/ 

0.854 

0.925 

High HS GPA (3.5-4.0) 



/ 

0.971 

0.869 

Cumulative GPA up to First Summer 


\/ 


1 467*** 

Cumulative Credit Earned up to First Summer 




1.030*** 

Took Summer Bridge Program 




1 973 *** 

Full-time in Fall 2003 




0.826 

Full-time in Spring 2004 




1.231 

Number of Observations 


4,280 

4,280 

4,280 

Pseudo R-square 


.029 

.033 

.102 


*** pcO.Ol, ** p<0.05, * p<0.1 

Note: Pseudo R-square is not valid for logistic regression that use replicate survey weights, so we have 
computed it instead using sample weights. 
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Table 3. Logistic regressions and Propensity-Matched Models Predicting Effect of Summer 
School on Retention and Graduation for students who entered two-year colleges. 

Comparing the percentages of summer attendees and non-attendees on various outcomes: 

Conditioned on fall 2004 enrollment 


Regression Model (%) Propensity Model (%) 



Summer 

No 

Difference 

Summer 

No 

Difference 



Summer 



Summer 


Outcome 







Reenrolled Spring 2 nd Year 

86.15 

79.17 


88.08 

84.89 

3.18 

Ever Stopped Out 

30.86 

41.10 

10.24*** 

29.88 

33.39 

3.50 

Graduated within 6 years 

45.83 

36.04 

9 79 *** 

53.28 

42.30 

10.98*** 

N 


3,020 



2,960 



*** p<.001; **p<.01; *p<.05 

These models include controls for students’ demographic and family background, high school academic 
preparation, academic performance during the first year of college, and other factors. See text for a 
complete list. 
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Table 4. Heterogeneity Tests of Summer Effects on Graduation among Two-year College 
Students Conditioned on fall 2004 enrollment 


Graduated in 6 years 

Treated Controlled Difference S.E. P P of 

differences 

between 

_ groups 

Age (mean=20.85, median-19) 


Younger (younger than 21) 

.4327 

.3254 

.1072 

.0424 

.011 


Older (21 or older) 

.5790 

.4754 

.1036 

.0289 

.000 

.944 

Race 








Minority (Black and Hispanic) 

.4142 

.2454 

.1688 

.0435 

.000 


Non-Minority (White, Asian, 

.5708 

.4601 

.1106 

.0280 

.000 

.260 

Others) 

Gender 








Male 

.5398 

.4166 

.1231 

.0395 

.001 


Female 

.5263 

.4206 

.1057 

.0307 

.000 

.728 

Remediation 








Took at least one remediation 

.4929 

.3813 

.1116 

.0284 

.000 


Didn’t take any remediation 

.6503 

.5437 

.1066 

.0512 

.037 

.932 

Family background 








1 st generation college-goer 

.5025 

.3884 

.1141 

.0338 

.000 


Not a 1 st generation college-goer 

.5623 

.4331 

.1292 

.0368 

.000 

.762 

Cumulative GPA up to First 








Summer 








Low GPA (GPA is Below 2.97) 

.3360 

.2560 

.0800 

.0362 

.027 


High GPA (GPA is 2.97 or 

.6312 

.5091 

.1220 

.0304 

.000 

.374 

higher) 








Propensity of Taking First Summer 







Low Propensity 

.4690 

.3756 

.0933 

.0381 

.014 


High Propensity 

.5478 

.3933 

.1544 

.0296 

.000 

.205 
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Table 5. Attendance in Summer Courses after the first year of college: Entrants to Four-year 
Colleges (weighted) 

Odds ratios from logistic regressions predicting First Summer Attendance. 



(1) 

(2) 

(3) 

Variables 

Demographic 

Academic 

Academic 


and SES 

Background 

Background 



in High 

in College up 



School 

to First 




Summer 


Black {Ref: white) 

Latino 
Asian 
Other race 
Female 

Age (centered) 

Age A 2 (centered) 

HH Income (log) 

Parents' Education is Unknown ( Ref: Parent HS) 

Parents LT Bach 

Parents have BA or higher 

Home Ownership 

Total amount of need-based grant (in thousand$) 
Married 

Have Any Dependent 

English as Primary Language 

No High School GPA Info {Ref: GPA 3.5-4.0) 

Low HS GPA(0.5-1.9) 

Medium-low HS GPA(2.0-2.9) 

Medium-high HS GPA(3.0-3.4) 

Cumulative GPA up to First Summer 
Cumulative Credit Earned up to First Summer 
Taking Summer Bridge 
Full-time in Fall 2003 
Full-time in Spring 2004 

Observations 

Pseudo R-square _ 

*** p<0.01, ** p<0.05, * p<0.1 


1.342* 

1.401** 

1.345* 

1.156 

1.192 

1.189 

2.092*** 

2.031*** 

I. 947 *** 

1.009 

1.009 

1.033 

1.193** 

1.168** 

1.118 

1.005 

0.992 

0.998 

1.001 

1.001 

1.000 

0.956 

0.957 

0.954 

0.779 

0.773 

0.748 

1.013 

1.005 

0.986 

1.221 

1.194 

1.158 

1.043 

1.048 

1.032 

0.962*** 

0.961*** 

0.961*** 

1 . 866 * 

1.869* 

1.884* 

1.155 r 

1.140 

1.113 

0.649*** 

0.658*** 

0.664*** 


1.064 

1.038 


1.847 

1.885 


0.724** 

0.809 


0.856* 

0.926 



1.232*** 



0.998 



2.676*** 



0.928 



1.098 

7,390 

7,390 

7,390 

0.023 

0.026 

0.040 
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Table 6. Logistic regressions and Propensity-Matched Models Predicting Effect of Summer 
School on Retention and Graduation for students who entered four-year colleges. 


Comparing the percentages of summer attendees and non-attendees on various outcomes: 

Conditioned on fall 2004 enrollment 



Regression Model (%) 

Propensity Model (%) 


Summer 

No 

Difference 

Summer 

No 

Difference 



Summer 



Summer 


Outcome 







Reenrolled Spring 2 nd Year 

94.83 

93.92 

0.91 

95.87 

93.78 

2.08** 

Ever Stopped Out 

14.49 

16.37 

1.88 

14.16 

17.03 

-2.86 

Graduated in 6 year 

73.94 

70.20 

3.74* 

76.40 

69.47 

6.93* 

N 


6,620 



6,600 



*** p<.001; **p<.01; *p<.05 

These models include controls for students’ demographic and family background, high school academic 
preparation, academic performance during the first year of college, and other factors. See text for a 
complete list. 
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Table 7. Heterogeneity Tests of Summer Effects on Graduation among Four-year College 
Students 

Conditioned on fall 2004 enrollment 


Graduated in 6 years 



Treated 

Controlled 

Difference S.E. 

P 

Difference 

of 

differences 

Age (mean -19; median-18) 








Younger (younger than 21) 

.7816 

.7155 

.0661 

.0158 

.000 


Older (21 or older) 

.5000 

.5131 

-.0131 

.1232 

.914 

.524 

Race 








Minority (Black and Hispanic) 

.6405 

.5693 

.0711 

.0418 

.089 


Non-Minority (White, Asian, 

.7946 

.7441 

.0505 

.0166 

.002 

.646 

Others) 








Gender 








Male 

.7113 

.6501 

.0612 

.0267 

.022 


Female 

.7953 

.7396 

.0556 

.0192 

.003 

.865 

Remediation 








Took at least one remediation 

.6592 

.5930 

.0661 

.0298 

.026 


Didn’t take any remediation 

.8201 

.7756 

.0444 

.0176 

.011 

.530 

Family background 








1 st generation college-goer 

.7113 

.6086 

.1026 

.0330 

.001 


Not a 1 st generation college-goer 

.7877 

.7329 

.0548 

.0180 

.002 

.203 

Cumulative GPA up to First 








Summer 








Tow GPA tGPA is Below 3.(0 

.6788 

.051 / 

.1270 

.0248 

.000 


High GPA (GPA is 3.0 or higher) .8416 

.8356 

.0059 

.0184 

.746 

.000 

Propensity of Taking First Summer 







Low Propensity 

.7862 

.7001 

.0861 

.0234 

.000 


High Propensity 

.7510 

.7121 

.0389 

.0214 

.068 

.136 


Table 8. Sensitivity Analysis for the Effect of Summer School on Graduation 

Conditioned on fall 2004 enrollment, ATT 




2 year college 


4 year college 

Estimated Difference 


10.98% 


6.93% 


Gamma 

P-value 

Gamma 

P-value 


1 

P<.000 

1 

P<.000 


1.1 

P<.000 

1.05 

.003 


1.2 

P<.000 

1.1 

.020 


1.3 

.004 

1.15 

.072 


1.4 

.041 

1.2 

.185 
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1.5 

.185 

1.25 

.360 

1.6 

.456 

1.3 

.466 

1.7 

.292 

1.35 

.280 

1.8 

.107 

1.4 

.144 

1.9 

.028 

1.45 

.064 

2 

.005 

1.5 

.024 
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Table 9. Predictors for the Propensity Score Matching Model 


Demographic background 

race*female, age, dependency status, single parent status, 
citizenship status, has any dependents, whether the 
student’s native language is English, household size, 
housing status (whether a student live on campus, off 
campus, or living with parents), degree of urbanization 

SES background 

mother and father’s educational level, non-traditional 
household status, homeownership status, 
homeownership*dependency status, whether parents or 
students have significant investments status, investment 
status*dependency status, income (log), 
income*dependency status, percentage of loan out of 
total aid 

Students’ high school academic 
preparation 

SAT math quartile, whether students earned any college 
level credits while in high school, high school GPA, 
whether they lacked a regular high school diploma, 
highest level of mathematics taken in high school 

College characteristics 

institutional enrollment size (log), percent received 
federal grants at institution, percentage of black and 

Latino at institution, tuition quartile, institutional sector 

Student academic progress and 
engagement during the freshman 
year of college 

whether the student viewed themselves as primarily a 
student working to pay expenses or primarily a worker 
who was taking courses in college, whether the student 
had paid work during the summer, hours worked per 
week while enrolled in 2004, whether the student 
registered full-time or part-time, whether the student took 
a summer bridge course prior to the Fall semester of their 
first year, the student’s cumulative credits earned and 
cumulative GPA during the first year prior to summer 
school, the ratio of credits completed to credits attempted 
in the first year of college 


Note: For the analyses of 4-year college students, a measure of college selectivity was added to 
these predictors in the propensity models. 
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Table 10. Statistics for Propensity Score Matches involving Two-year students 



N 

% 

Matched 

% Reduction 
in Bias after 
matching 

P-value of T 
Test for 
Treat-Control 

Mean Bias 
of covariates 
after 

matching 

Table 3 (ATT: In condition of 2,960 99.46 99.9 

fall 2004 enrollment) 

Table 4 (ATT: In condition of fall 2004 enrollment, Heterogeneity Tests) 
Age 

0.990 

.025 

Younger (younger than 21) 

739 

96.34 

99.9 

0.994 

.038 

Older (21 or older) 

Race 

2,186 

98.95 

99.9 

0.985 

.033 

Minority 

(Black and Hispanic) 

776 

95.33 

99.8 

0.977 

.034 

Non-Minority (White, Asian, 
Others) 

Gender 

2,156 

99.72 

99.9 

0.989 

.025 

Male 

1,203 

99.91 

100 

0.997 

.031 

Female 

1,757 

99.15 

99.8 

0.973 

.029 


Remediation 

Took at least one remediation 
Didn’t take any remediation 
Family background 
1 st generation college goer 
Not a 1 st generation college 
goer 

Cumulative GPA up to First 
Summer 

Low GPA (GPA is Below 
2.97) 

High GPA (GPA 2.97 or 
higher) 

Propensity of Taking First 
Summer 
Low Propensity 
High Propensity 



1,606 97.15 


99.9 


0.960 

0.971 

0.986 

0.971 


0.990 

0.977 


0.995 

0.988 


.037 

.058 

.040 

.037 


.040 

.026 


.030 

.026 
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Table 11. Statistics for Propensity Score Matches involving Four-year students 


N 

% 

% Reduction 

P-value of T 

Mean Bias 


Matched 

in Bias after 

Test for 

of covariates 



matching 

Treat-Control 

after 

matching 


6,598 

99.93 

100 

0.997 

.023 

2004 enrollment, Heterogeneity Tests) 



236 

93.28 

99.5 

0.950 

.096 

6,341 

100 

99.9 

0.990 

.023 

1,106 

99.01 

100 

1.000 

.030 

5,483 

99.96 

99.9 

0.989 

.018 

2,817 

99.75 

99.9 

0.992 

.027 

3,775 

99.92 

100 

1.000 

.024 

2,061 

99.56 

99.8 

0.980 

.023 

4,524 

99.82 


99.9 

1 

0.992 

.021 

1,791 

99.72 


99.9 

0.992 

.033 

4,748 

99.93 


100 

1 

1 

0.996 

.019 


Table 6 (ATT: In condition of 
fall 2004 enrollment) 

Table 7 (ATT: In condition of fall 
Age (mean =19; median=18) 
Older (19 or older) 

Younger 

Race 

Minority 

Non-Minority 

Gender 

Male 

Female 

Remediation 

Took at least one remediation 
Didn’t take any remediation 
Family background 
1 st generation college goer 
Not a 1 st generation college 
goer 

Cumulative GPA up to First 
Summer 
Low GPA 
High GPA 

Propensity of Taking First 
Summer 
Low Propensity 
High Propensity 


3,058 

3,526 


3,273 

3,321 


99.70 

99.74 


100 

99.87 


99.9 

99.9 


99.8 

99.9 


0.996 

0.990 


0.987 

0.994 


.026 

.026 


.021 

.025 
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